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I.  Introduction 


There  seems  to  be  an  increasing  reliance  on  the  use  of  , 

surveys  to  collect  data  to  be  used  in  a wide  variety  of  research 
settings.  In  many  cases  the  research  results  are  to  be  used  to 
derive  (or  support)  policy  decisions.  The  questions  which  arise, 
and  which  motivate  these  remarks,  are;  "Is  survey  research 
necessary?";  "Is  it  cost-effective?";  "What  alternatives  exist?"; 
and  "How  might  they  be  done  better,  when  necessary?". 

II.  Economic  Research  and  Use  of  Survey  Methods 

There  exist  two  major  ways  economists  analyse  "what  people 
do"  in  order  to  predict  what  their  future  economic  behavior  will 
be: 

a)  They  assume  that  there  exists  a utility  function  with 
certain  properties  which  embody  the  important  decision  variables 
and  parameters,  and  which  relates  their  combination  to  levels  of 
consumer  satisfaction;  then  they  predict  on  the  basis  of  changes 
in  parameters  through  analytical  manipulation  of  mathematical 
models  based  on  these  functions. 

b)  They  assume  that  people  behave  rationally,  or  at  least 
consistently,  and  observe  what  people  do,  or  did,  and  therefore 

derive  predictions  of  what  they  will  do  and  how  well  off  they  will  i 

be,  comparatively,  when  parameters  change.  This  is  the  "revealed 
preference"  approach. 

In  the  former  case,  we  may  use  the  theory  of  consumer  be- 
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havior  so  developed  to  construct  testable  hypotheses  of  the  way  | 

they  will  behave  in  the  future,  and  judge  the  "validity"  of  the  | 

underlying  assumptions  and  theoretical  structures  (e.g.,  "forms"  • 
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of  utility  functions)  on  the  basis  of  the  actually  observed  be- 
havior. In  this  case  one  may  argue  that  surveys  taken  before 
parametric  changes  in  one's  environment  occur,  or  before  decisions 
are  made  even  without  such  changes,  will  provide  "ex  ante"  in- 
formation which  may  enable  the  theoretical  structures  to  be  modified 
and  thus  be  better  predictors  of  results  than  if  events  are  merely 
"allowed"  to  happen. 

However  economists  have  been  traditionally  skeptical  of  the 
value  of  many  such  survey  techniques  because  of  their  knowledge 
of  the  basic  incentives  which  are  believed  to  motivate  behavior. 

For  example,  many  experiments  have  demonstrated  the  failure  of 
the  "what  if"  type  hypothetical  choice  results  to  conform  to  the 
ubiquitous  utility  theory  assumption  of  "Transitivity  of  Preferences". 
The  rationale  is  that  since  the  subject  consumers  did  not  actually 
have  to  make  and,  live  with,  their  choices,  their  incentives  were 
insufficient  to  get  them  to  make  the  true  choices  (i.e.,  the  ones 
they  really  would  have  made  in  such  an  actual  choice  situtation) . 

An  additional  confounding  factor  occurs  in  situations  where  the 
goods  or  services  under  study  are  not  pure  "private  goods"  but 
rather  exhibit  varying  degrees  of  externalities  and/or  are  provided 
by  the  government  (at  some  level)  as  "public  goods".  The  problem 
here  is  simple:  If  the  amount  of  a good  or  service  to  me  is  not 
diminished  by  your  consumption  of  it,  I'd  rather  that  you  pay  for 
it,  or  that  you  be  responsible  for  its  existence  rather  than  me.  I 
have  an  incentive  to  hide  (to  different  degrees)  my  true  preferences 
in  such  cases. 

Next  consider  the  revealed  preference  approach.  If  we  wait 
until  behavior  is  actually  observable  to  attempt  relative  utility 
calculations  [not  between  consumers,  but  for  an  average,  or  subset 
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of,  consumer Cs) ] , we  have  lost  the  ability  to  influence  these 
utility  levels.  In  addition,  many  other  variables  will  have  also 
changed,  making  it  difficult  to  lay  the  credit  or  blcime  for  changes 
in  satisfaction  on  the  targeted  policy  variable.  Likewise  if  we 
collect  only  data  from  past  actions,  we  may  have  too  few  data 
points  from  which  to  generalize  and  the  ceteris  paribus  conditions 
may  also  have  been  violated.  Thus  although  it  would  seem  that 
surveys  could  facilitate  collection  of  more  data  points  and  then 
enhance  the  probability  of  accurate  predictions,  even  when  based 
in  large  part  on  the  revealed  preference  approach,  the  same  ob- 
jections as  noted  above  regarding  the  utility  theory  approach 
would  then  hold  in  this  case. 

For  these  reasons  economists  have  long  been  hesitant  to  rely 
in  any  significant  way  on  survey  data  in  the  future  sense  of  the 
word,  that  is,  where  hypothetical  situations  and  questions  are 
analysed.  We  as  a profession  have  been  more  aunenable  to  the  use 
of  surveys  to  collect  data  dealing  with  the  present,  i.e.,  factual 
augmentation,  when  files  and  records  are  insufficient.  However, 
some  economists  certainly  have  based  their  research  efforts  on 
survey  work.  The  recognized  leader  among  economists  in  the  develop- 
ment and  use  of  survey  data  is  George  Katona,  who  until  recently 
was  research  coordinator  at  the  Survey  Research  Center  at  the 
University  of  Michigan.  His  position  and  proclivities  have  led 
him  to  differ  significantly  with  the  majority  of  economists  in 
assessing  the  purpose  and  utility  of  survey  research.  He  has  said 
that  "The  primary  purpose  of  surveys  are  to  obtain  information  on 
trends  over  time  and  a functional  relationship  emong  variables. 
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This  statement  has,  first,  a negative  implication:  Determination 
of  a position  as  it  prevails  at  a given  time  is  not  the  major 
purpose  of  surveys  ....  Such  findings  do  not  constitute  major  goals 
of  survey  research  and  are  subject  to  large  errors  ....  Absolute 
measurements  represent  but  the  starting  point  for  studies  in 
Behavioral  Economics"  [3,  p.  409-410]. 

Katona's  assertion  that  "Studies  of  changes  in  attitudes  and 
expectations  and  their  relation  to  behavior  call  for  the  collection 
of  new  data  [through  surveys]",  is  tempered  by  his  further  state- 
ment that  "field  observations  and  experiments  may  supplement  the 
survey  method"  [p.  405] . The  major  assertion  however  is  consistent 
with  the  pioneering  work  done  over  the  years  by  the  Survey  Research 
Center  in  providing  information  on  the  attitudes  and  intentions  of 
consumers  and  business  decision  makers  which  has  proven  extremely 
useful  in  formulating  macro-economic  policy. 

In  my  view  the  situation  is  not  clear  cut  at  all.  As  is 
usually  the  case,  the  use  of  survey  versus  non-survey  methods  to 
collect  information  and  data  in  a variety  of  problem  settings  should 
be  determined  by  the  problem  and  its  setting,  and  generally  will  in- 
volve some  judiciously  chosen  mix  of  the  two  methodologies.  Addition- 
ally one  must  distinguish  between  surveys  which  collect: 

a)  Data  on  physically  existing  facts  (e.g.,  the  shelf- 
prices  at  stores)  which  do  not  exist  in  records  or  files; 

b)  Data  on  non-manipulable  individual  characteristics 
(e.g. , demographics) ; 

c)  Data  which  requires  a person  to  recall  behavior  or 
opinions  from  the  past; 

d)  Data  on  probable  future  behavior  or  expectations. 
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Certainly  the  majority  of  economists  have  few  qualms  about 


the  first  two  uses  of  surveys.  For  example,  in  my  own  research, 
we  have  surveyed  the  catchment  population  eligible  to  use  military 
health  care  facilities  to  ascertain  the  age/sex  beneficiary  status 
in  order  to  derive  subgroup  utilization  rates.  Such  data  does 
not  exist  in  the  presently  structured  military  health  care  system. 

The  problems  with  these  sorts  of  surveys  are  mostly  technical  — 
i.e.,  selecting  the  correct  sample  size  and  its  composition  to  ob- 
tain reasonable  estimates,  and  are  very  amenable  to  solution. 

However  there  are  more  serious  problems  with  the  latter  two 
types  of  surveys.  Many  times  people  cannot  accurately  recall  their 
behavior  or  facts  about  a previous  situation.  Added  to  this  are 
the  incentive  problems  mentioned  above  associated  with  motivating 
people  to  answer  questions  about  their  future  plans/behavior  truth- 
fully. In  addition,  Katona  has  noted  that  "Survey  research  is  most 
reliable  if  it  is  directed  toward  obtaining  information  on  variables 
that  are  widely  represented  in  the  universe",  [p.  421]  , agreeing 
that,  "unlike  laboratory  experiments,  surveys  cannot  manipulate 
independent  variables"  Ip.  423]  . 

What  then  would  I recommend  when  it  comes  right  down  to  the 
question  of  whether  we  need,  or  should  use,  a suirvey  methodology? 

Based  on  my  experience  and  education  it  seems  that  we  have  come  to 
the  juncture  of  survey  and  experimental  methods.  I believe  that 
what  is  required  is  a damper  on  the  "knee-jerk"  propensity  to  write 
out  a survey  instrument  everytime  we  need  some  data  and  on  the  equally 
habitual  reluctance  to  Incorporate  any  survey  method  or  data  into 
research  projects.  Our  best  bet  is  to  integrate,  in  an  innovative 
and  appropriate  manner,  survey  methods  where  the  payoff  warrants  it. 
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The  questions  we  may  ask  are; 


a)  How  should  this  recommended  innovative  integration  of 
survey  and  non-survey  research  be  determined?;  and 

b)  How  is  it  being  done  in  ^iconomic  research  at  the  present? 

In  my  opinion,  the  basic  question  should  be,  "What  am  I try- 
ing to  do,  and  what  are  the  feasible  ways  to  do  it,  and  their  re- 
spective costs  and  benefits,  both  monetary  and  non-monetary?".  Each 
of  us  does  this  implicitly  when  we  seek  data  and  information  for  our 
own  use.  For  example,  when  searching  for  a place  to  buy  film  for 
our  camera,  we  are  likely  to  base  our  decision  on  past  experience 
and  a casual  perusal  of  the  newspaper  ads.  However,  when  searching 
for  a camera,  we  may  ask  friends,  known  to  have  purchased  one,  where 
they  bought  it  and  if  they  felt  it  was  a "good  buy" . When  buying  a 
specific  make  of  car,  we  however  tend  to  rely  more  heavily  on  rig- 
orously collected  price  data,  and  may  only  supplement  this  with  a 
friend's  opinion.  Clearly  the  greater  the  cost  of  making  a wrong 
decision,  the  more  accurate  and  objective  we  want  our  data  to  be. 

The  collective  experiential  sxarvey  data  of  Consumer  Reports , cost- 
effectively  available,  is  preferable  to  a small  sample  of  friends' 
opinions  for  major  purchase  decisions.  However,  we  would  not  indi- 
vidually attempt  to  collect  such  data  if  Consumer  Reports  was  not 
available.  It  would  not  be  worth  it. 

In  research  projects  whose  outcome  is  to  be  a specific  policy 
recommendation,  such  costs  and  benefits,  in  a probablistic  sense, 
should  be  assessed  during  the  research  methodology  development 
stage.  The  expected  value  of  each  feasible  strategy  will  depend  on 
an  assessment  of  the  relative  accuracy  and  reliability  of  the  data 
to  be  so  obtained.  These  factors  must  be  explicitly  considered. 
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There  is  no  way  to  generalize  a specific  algorithm  in  such  cases 
because  there  are  innumerable  unique  potential  situations.  How- 
ever, below  I describe  a couple  of  examples  of  recent  research  pro- 
jects which  illustrate  my  points. 

Ill . Innovative  Use  of  Modified  Survey  Methodologies  in  Economic 
Research 

As  mentioned  earlier,  there  is  a major  problem  in  using 
survey  research  in  cases  where  those  surveyed  have  an  inherent  in- 
centive to  hide  their  true  feelings  or  opinions  (as  contrasted  with 
situations  where  there  are  merely  insufficient  incentives  to  assess 
them  or  where  significant  uncertainty  exists) . Thus  if  in  voting 
on  a bond  issue  we  incorporated  a space  for  those  who  vote  "yes" 
to  indicate  how  much  they  would  be  willing  to  pay  toward  the  cost  of 
providing  the  facility  under  consideration,  we  would  expect  that 
individuals  who  favor  the  issue  would  either  vote  "no"  hoping  that 
enough  others  would  vote  "yes"  and  be  assessed,  or  would  vote  "for" 
the  issue  but  significantly  understate  the  value  they  place  on  having 
the  facility  hoping  to  minimize  the  amount  of  additional  tax  they 
personally  would  have  to  pay. 

This  phenomena  has  caused  great  difficulty  in  the  past  in 
that  we  collectively  have  had  to  rely  upon  indirect  assessments  — 
i.e.,  everyone's  tax  would  go  up  (if  approved)  by  the  same  marginal 
increment.  This  results  in  some  public  goods  not  being  provided 
even  though  a sufficient  number  of  voters  might  be  willing  to  pay 
differing  amounts  to  finance  it,  because  a majority  were  not  willing 
to  pay  the  additional  stated  tax  increment.  Likewise,  even  when 
bond  issues  pass,  all  this  indicates  is  that  enough  voters  were 
willing  to  pay  the  stated  "price",  not  that  the  optimal  relative  tax 
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burden  has  been  defined. 

When  voting  or  otherwise  expressing  preferences  for  non-local 
public  goods,  most  times  there  is  little  direct  connection  between 
passage  of  such  an  issue  and  the  cost  to  the  voter.  Thus,  the 
. voters  or  respondents  may  tend  to  indicate  they  favor  the  adoption 

of  some  policy  because  they  perceive  the  cost  to  them  to  be  negli- 
' gible,  or  fail  to  perceive  the  opportunity  cost  of  passage. 

Thus  research  into  the  desirability  of  producing  or  providing 
such  public  goods  has  concentrated  on  the  technological  aspects  and 
the  associated  costs  of  producing  the  goods  or  services  and  to  a 
lessor  extent  on  "a  priori"  utility  assessments  using  utility  theory 
and  revealed  preference.  The  benefits  and  opportunity  cost  asses- 
sments tend  to  be  left  to  be  debated  by  opposing  sides  and  are 
assumed  to  be  communicated  to  the  population  of  voters  by  the  media. 
The  consumer  opinions  are  then  "collected"  either  through  monitoring 
constituent  mail  or  by  allowing  an  eventual  popular  vote  on  the 
issue,  on  an  either/or  basis  only.  Although  polls  may  be  taken  to 
assess  the  general  climate  of  consumer  opinion,  the  polls  are  not 
used  (generally)  to  determine  the  policy  decision  directly. 

Groves  and  Ledyard  have  developed  a research  methodology  [ 2 . ] 
which  is  posited  to  be  a useful  first  step  in  changing  this  process. 
They  have  designed  a "preference  revelation"  process,  with  feedback 
to  the  "voter",  which  elicits  the  subjects  valuation  of  the  project 
under  consideration.  The  major  element  present  here  is  a penalty/ 

. reward  structure  for  the  voter  as  a result  of  an  indicated  decision 

or  value  assessment.  While  the  details  are  too  complex  to  present 
here,  the  salient  point  for  our  purposes  is  that,  recognizing  the 
c inherent  difficulties  in  using  a standard  survey  for  value  assessment, 
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they  have  attempted  to  construct  an  incentive  mechanism  tied  to 


truthfulness  on  the  survey  form  to  ensure  greater  accuracy  of  the 
data  collected  for  decision-making.  The  results  of  such  a perfected 
methodology  would  facilitate  a benefit  calculation  to  accompany  the 
cost  estimates  of  specific  public  good  type  projects  and  would 
allow  those  projects  to  be  chosen  which  yield  maximum  benefits  as 
revealed  by  the  population  and  which  would  be  financed  in  a pre- 
determined non-homogenous  way. 

Groves  has  also  related  this  "Teeim  Theory"  work  to  the  problem 
of  managing  a decentralized  firm  II.].  It  appears  that  this  same 
sort  of  quasi-survey  approach  with  incentives  may  well  have  applica- 
tion to  the  management  of  the  Military  Health  Care  Delivery  System 
under  Capitation  Budgeting  with  accompanying  structural  changes. 

The  potential  for  the  adoption  of  a universal  enfranchisement 
plan  — i.e..  National  Health  Insurance  (NHI)  — to  significantly 
alter  the  existing  utilization  patterns  by  individuals  of  the  overall 
Health  Care  Delivery  System  has  made  it  difficult  to  estimate 
accurately  the  cost  of  the  various  competing  NHI  proposals  before 
the  congress.  Our  experience  with  the  rapid  cost  escalation  ac- 
companying the  adoption  of  Medicare  and  Medicaid  has  caused  signifi- 
cant concern  and  motivated  a desire  to  know  more  exactly  the  probable 
financial  consequences  of  NHI  implementation. 

Existing  (natural  experimental)  data  is  not  sufficient  to 
accomplish  these  projections  because  of  the  significant  differences 
in  the  coverages  and  real  prices  of  health  care  to  individuals  under 
NHI  vis-a-vis  the  current  patterns  of  insurance  and  care.  Likewise, 
the  intricate  nature  of  the  decision  process  which  leads  to  the 
observation  of  a demand  for  care  and  the  previously  discussed 
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"public  good"  aspect  associated  with  NHI  [i.e.,  if  NHI  passes, 
everyone  has  it  not  just  those  who  might  be  willing  to  pay]  are 
among  the  many  reasons  why  a straight-forward  survey  of  individuals 
to  ask  their  probable  future  utilization  under  various  plans  is  an 
inappropriate  method  to  provide  answers. 

The  Rand  NHI  study,  a $32  million  project  to  measure  and 
predict  the  effects  of  price  changes  on  utilization  of  health  care 
services  was  thus  designed  as  a mix  of  an  experimental,  randomized 
clinical  trails  type  strategy  and  a survey  component.  This  measure- 
ment of  the  price  effect  is  not  simple.  "Measurement  of  the  effect 
of  price  on  demand  can  be  refined  in  several  ways.  For  example,  we 
may  wish  to  know  whether  the  effect  of  price  varies  with  income  and 
total  expenditure.  Do  poor  families  respond  more  readily  to  price 
than  affluent  ones?  If  so,  they  will  benefit  differentially  from  a 
generous  health  insurance  plan.  Do  families  with  major  illnesses 
respond  differently  from  those  with  minor  illnesses?  If  so,  esti- 
mates of  the  effect  of  catastrophic  health  insurance  must  be  based 
only  on  data  from  those  with  such  illnesses"  [4.,  p.  6].  Further, 
given  that  we  are  interested  in  effects  of  price  changes,  changes 
in  coverage  must  also  be  evaluated.  "To  compute  this  change,  one 
must  know  the  current  coverage  of  the  population;  unfortunately, 
detailed  information  on  this  subject  is  rare.  National  Health 
Survey  estimates  can  be  used  to  determine  the  percentage  of  the 
population  with  no  coverage,  but  they  are  not  helpful  if  one  wants 
to  determine  the  improvement  that  would  be  caused  by  any  particular 
plan. . . .eunong  persons  who  now  have  partial  coverage  (as  most  do)" 
[4.,  p.  6].  The  Rand  study  had  to  fill  these  gaps,  since,  for 
example. .so  few  of  the  respondents  know  the  details  of  their 
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policy  that  the  National  Health  Survey  has  stopped  asking  such 
questions"  [4.,  p.  9]. 

After  painstaking  analysis,  the  Rand  group  designed  a 
5 year  experiment  in  which  3200  families  from  4 areas  will  be 
randomly  enrolled  in  variously  structured  health  plans  which  , 

systematically  vary  the  net  price  for  health  care  they  face. 

Their  utilization  and  expenditure  experiences  will  be  carefully 
and  exhaustively  monitored  ($5  million'  of  the  budget  is  for 
computer  charges) . Although  most  of  the  data  will  be  collected 
through  non-survey  methods,  surveys  do  play  a significant,  and 
integral  role  in  the  study. 

For  example,  in  the  participant  selection  process,  6,000 
families  from  each  site,  selected  from  clustered  random  population 
samples  were  given  screening  interview  surveys.  Subsequently 
2,000  families  were  chosen  for  lengthy  personal  surveys  in  accord- 
ance with  the  Morris  Finite  Selection  Model  on  the  basis  of  the 
information  gained  from  the  initial  screens.  Then,  based  on  the 
more  detailed  family  specific  data  gathered  in  the  lengthy  surveys, 

800  families  were  selected — 500  for  enrollment  in  the  experimental 
group,  and  300  for  the  control  group  using  the  Conlisk-Watts  Model 
to  choose  and  distribute  the  families  among  the  various  plans  to 
minimize  non-random  variation  in  the  utilization  experiences  to 
be  observed.  Finally,  quarterly  surveys  of  the  participants  will 
provide  supplemental  and  valuation  data  to  that  collected  directly 
during  the  experiment.  • 

Although  there  have  been  many  questions  raised  about 
specific  aspects  of  the  study,  it  seems  a fair  evaluation  to  state 
that  the  study  group,  on  the  basis  of  rigorous  analysis  of  the 
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potential  biases  and  other  problems  with  the  data  likely  to  result, 
has  taken  a significant  step  toward  providing  the  most  useful  demand 
data  yet  available,  as  well  as  providing  an  exeimple  of  an  innovative 
mix  of  our  two  subject  research  methodologies. 

IV.  My  View  of  the  Future 

It  seems  clear  to  me  that  the  most  valuable  empirical  research 
in  the  future  will  utilize  a judicious  combination  of  survey  and 
experimental  methodologies.  The  character  of  the  major  social 
economic  problems  facing  us  and  the  fact  that  in  our  ever  more 
fine-tuned  economy  mistakes  may  impose  long  term  penalties  we  can 
ill  afford  to  pay,  seems  unambiguously  to  point  to  economic  research 
which  integrates  the  probable  reactions  of  consumers  to  a greater 
extent  than  ever  before.  The  previous  predilection  of  economists 
(and  others)  to  see  the  problem  in  a "partial  equilibrium"  setting 
is  slowly  giving  way  to  a broader  perspective  in  structuring  research 
to  solve  seemingly  micro  scale  problems.  This  will  not  always  in- 
volve survey  research,  and  in  fact  in  many  instances,  may  involve 
less  (or  no)  such  methods  in  cases  where  they  previously  had  been 
used.  But  certainly  survey  research  will  provide  significant  bene- 
fits, when  carefully  tailored  to  the  problem  at  hand. 

A final  example  may  best  illustrate  my  point.  Without  going 
into  lengthy  detail,  many,  including  myself,  have  recommended  that 
the  Military  Health  Services  System  be  significantly  restructured 
to  decentralize  decision  making  and  provide  greater  incentives  for 
cost  containment.  The  major  elements  of  such  a system  are  the  use 
of  capitated  budgets  for  facilities  and  their  providers,  increased 
authority  for  local  decision  makers,  regional  budgets  and  resource 
responsibility,  sharing  of  the  fruits  of  cost  savings  with  health 


care  providers  and  staffs,  and  teventual)  dual  choice  of  enrollment 
for  the  eligible  population.  The  view  of  the  problem  taken  is  thus 
clearly  holistic  and  any  recommendations  for  a specific  implementa- 
tion strategy  and  associated  pilot  tests  involves  many  facets.  In 
particular  we  recognized  that  allowing  "dual  choice"  could  have 
potentially  catastrophic  fiscal  implications  if  many  people  decided 
to  use  non-direct  facilities  and  providers  (i.e.,  through  CHAMPUS) 
as  their  source  of  care.  Likewise,  we  realized  that  giving  the  pro- 
viders positive  incentives  for  cost  control  as  we  suggested,  could 
lead  to  less  acceptable  care  as  evaluated  by  patients,  and  thus 
further  exacerbate  the  outflow  of  patients  to  CHAMPUS  enrollment. 

It  was  thus  deemed  important  to  track  the  collective  patient  as- 
sessment of  the  health  care  delivery  process.  This  information  is 
possessed  only  by  the  patient  and  is  by  nature  extremely  subjective. 
Thus  the  only  way  to  obtain  this  data  was  to  ask  the  patients.  For 
this  reason  we  have  suggested  and  developed  a Patient  Satisfaction 
Monitoring  System.  This  relies  on  a random  sample  encounter -based 
survey  methodology  which  provides  feedback  to  the  clinic  level  staff 
and  indicates  areas  in  which  modifications  in  their  behavior  might 
be  productive.  This  information,  in  combination  with  the  internal 
systemic  incentive  changes  we  have  suggested,  will  tend  to  mitigate 
unnecessary  enrollment  turnover  and  hence  minimize  system  costs. 

The  point  is  that  the  decision  to  develop  and  use  such  a 
survey  methodology  was  made  after  an  analysis  of  the  overall  situation 
indicated  that  it  was  the  best  way  to  obtain  the  patient-subjective 
satisfaction  data  which  could  have  drastically  affected  system-costs, 
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and  hcnre  had  to  be  addressed.  Thus,  sanetimes  there  are  better 
alternatives  to  the  use  of  surveys,  sometimes  not.  The  choice  of 
the  tool  should  be  dictated  by  the  nature  of  the  problem — not  by 
habit  or  fiat. 
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